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CLAIMS 

I claim: 

1. A method for detecting an anomalous operation of a computer system, 
comprising: 

(a) monitoring transitions between and among program instrumentation points 
within an internal operating environment on the computer system and producing program 
execution trace dat^ 

(b) comparing the program execution trace data with data indicative of a 
nominal operation of the computer system; and 

(c) identifying an anomalous operation of the computer system based on the 
result of the comparison. 

2. A method as recited in claim 1, wherein said monitoring is performed by 
employing software signals obtained from instrumented code at instrumentation 
points in an execution path of the computer system. 

3. A method as recited in claim 1, wherein said monitoring is performed by 
employing software signals obtained from a hardware address bus associated with the 
computer system. 

4. A method as recited in claim 1, wherein said program execution trace data 
is employed to provide an execution profile including a list of execution paths that 
have executed in a specified time frame and the frequencies of executions. 

5. A method as recited in claim 1, wherein the computer system comprises a 
plurality of program modules in an instrumented software system. 

6. A method as recited in claim 5, wherein each program module implements a 
predefined fimctional requirement. 

7. A method as recited in claim 6, wherein each program module includes a 
mechanism for calling another module, and the method fiuiher comprises the use of a 
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statistical methodology to identify a relatively small set of cohesive program modules 
that represent the dynamic bindings among program modules as they execute. 



8. A method as recited in claim 7, wherein instrumentation points are 
employed to monitor the activity of an executing program and to indicate an epoch in 
the execution of the program. 

9. A method as recited in claim 8, further comprising recording, in an 
execution profile for ttie program, telemetry from the instrumentation points at each 
epoch. 

10. A method as recited in claim 9, wherein the execution profile comprises an 
n element vector (X) comprising at least one entry for each program module. 

1 L A method as recited in claim 10, wherein each element, , of said vector 
contains a frequency coimt for the number of times that the corresponding 

instrumentation point has executed during an era of A: epochs, where k-^x.. 

12. A method as recited in claim 11, wherein an execution profile is recorded 
whenever the number of epochs, A:, reaches a predefined count, at which time the 
contents of the execution profile vector is set to zero. 

13. A method as recited in claim 12, wherein the recorded activity of the 
program during its last L = JK epochs is stored in a sequence of / execution profiles, 
Xj,X2,. . .,X^. , where the value x- j represents the frequency of execution of the 
program module on the execution profile. 

14. A method as recited in claim 11, ftirther comprising the step of reducing 
the size of the execution profiles fromw, the number of instrumentation points whose 
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activity is highly correlated, to a smaller set of m virtual instrumentation points whose 
activity is uncorrelated. 



15. A method as recited in claim 14, wherein the statistical technique of 
principal components analysis is employed to reduce the dimensionality of the 
execution profiles. 

16. A method as recited in claim 14, wherein the statistical technique of 
principal factor analysis is employed to reduce the dimensionality of the execution 
profiles. 

17. A method as recited in claim 14, wherein an w x j\ j>n data matrix 

D = Xj,X2,...,Xy is factored into m virtual orthogonal module components, where m 
is less than n, whereby the dimensionality is reduced from n to m, 

18. A method as recited in claim 17, wherein an eigenvalue is associated 
with each of the m orthogonal components. 

19. A method as recited in claim 18, wherein the eigenvalues satisfy the 

n 

relation =«, 

1=1 

20. A method as recited in claim 17, further comprising using a predefined 
stopping rule in determining a number of components extracted in an orthogonal 
structure representing an execution profile with reduced dimensionality. 

21 . A method as recited in claim 20, wherem the stopping rule is: extract all 
components whose eigenvalues are greater that a predefined threshold. 
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22. A method as recited in claim 20, wherein the stopping rule is: extract those 

1 

components such that the proportion of variation represented by v = — V X is at least 
equal to a predefined value. 

23. A method as recited in claim 17, further comprising constructing a matrix 
(P), wherein said matrix is an « x m structure whose rows, p^j , contain values 
showing the degree of relationship of the variation of the f program module and the 

factor or principal component 

24. A method as recited in claim 17, further comprising the step of forming a 
mapping vector (O) for at least one execution profile vector. 

25. A method as recited in claim 24, wherein the mapping vector, Q 
comprises elements o^. whose values are defined as follows: 

max 

\<]<m 

let Oj = indtx(qj) represent the column number in which the corresponding 
value q. occurs. 

26. A method as recited in claim 25, wherein the mapping vector contains data 
to map probe event frequencies recorded in the execution profile vector onto 
corresponding virtual module equivalents. 

27. A method as recited in claim 26, wherein a fi-equency count for each 
instrumentation point k in an execution profile vector is represented by a value , 
and the mapping vector element Oj^ contains an the index value that A: maps into. 

28. A method as recited in claim 17, wherein m orthogonal sources of 
variation in the data vector D representing the original n program instrumentation 
points are identified. 



Page 28 of 38 



SOFT-0004 



PATENT 



29. A method as recited in claim 27, wherein, on each of the original raw 
execution profiles, the instnmientation point frequency count is represented in the 

elements, x^ j , of the profile vector, X^. . 

30. A method as recited in claim 24, wherein a frequency count for each 
instrumentation point A: in an execution profile vector is represented by a value ; 
wherein the mapping vector element o^^ contains an the index value that ^ maps into; 
wherein the mapping vector contains data to map probe event frequencies recorded in 
the execution profile vector onto corresponding virtual module equivalents; and 
wherein, after the mapping vector has been established, a virtual profile vector (X* ) is 

employed to contain the frequency counts for interactions among virtual execution 
domain sets. 

3 1 . A method as recited in claim 30, wherein the virtual profile vector, Y. , is 
defined by: 



32. A method for detecting an anomalous operation of a computer system 
including a plurality of program modules, comprising: 

(a) monitoring transitions between and among instrumentation points within an 
internal operating environment on the computer system, wherein said monitoring is 
performed by employing software signals obtained from instrumented code in the 
program modules; 

(b) providing program instrumentation trace data representative of the transitions 
between and among program modules within a time frame; 



n 




f Ofj!=k 

otherwise 
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(c) identifying a relatively small set of virtual execution domains whose 
activity is substantially uncorrelated, and using this information to reduce the amount 
of trace data needed to detect anomalous activity; 

(d) comparing the reduced amount of trace data with predefined data 

indicative of a nominal operation of the computer system; and 

(e) identifying an anomalous operation of the computer system based on the 
result of the comparison. 

33. A method as recited in claim 32, wherein said program execution trace 
data is employed to provide an execution profile including a list of execution paths 
that have executed in a specified time fi:ame and the frequencies of executions. 

34. A method as recited in claim 32, wherein each program module includes a 
mechanism for calling another module, and wherein step (c) comprises the use of a 
statistical methodology to identify a relatively small set of cohesive program modules 
that represent dynamic bindings among program modules as they execute. 

35. A method as recited in claim 32, wherein instrumentation points are 
employed to monitor the activity of an executing program and to indicate an epoch in 
the execution of the program. 

36. A method as recited in claim 35, further comprising recording, in a first 
execution profile for the program, telemetry firom the instrumentation points at each 
epoch. 

37. A method as recited in claim 36, wherein the first execution profile 
comprises an n element vector (X) comprising at least one entry for each program 
module, and wherein each element, , of said vector contains a frequency count for 
the number of times that the corresponding instrumentation point has executed 

n 

during an era ofk epochs, where k-^x.\ and wherein an execution profile is 
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recorded whenever the number of epochs, k, reaches a predefined count, K, at which 
time the contents of the execution profile vector is set to zero. 



38. A method as recited in claim 37, wherein the recorded activity of the 
program during its last L = jK epochs is stored in a sequence ofj execution profiles, 
Xi , X2 , . . . , X^. , where the value x^ j represents the firequency of execution of the f 
program module on the execution profile. 

39. A method as recited in claim 38, further comprising the step of reducing 
the dimensionality of the execution profiles from n, the number of instrumentation 
points whose activity is highly correlated, to a smaller set of m virtual instrumentation 
points whose activity is uncorrelated. 

40. A method as recited in claim 39, wherein the statistical technique of 
principal components analysis is employed to reduce the dimensionality of the 
execution profiles. 

41 . A method as recited in claim 39, wherein the statistical technique of 
principal factor analysis is employed to reduce the dimensionality of the execution 
profiles. 

42. A method as recited in claim 39, wherein an « x j\j>n data matrix 

D = XijXj,. . .,X^. is factored into m virtual orthogonal module components, where m 
is less than n, whereby the dimensionality is reduced from n to m. 

43. A method as recited in claim 42, wherein an eigenvalue A. is associated 
with each of the m orthogonal components. 

44. A method as recited in claim 43, wherein the eigenvalues satisfy the 

tt 

relation ^yl^ = n. 
/-I 
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45. A method as recited in claim 39, further comprising using a predefined 
stopping rule in determining a number of components extracted in an orthogonal 
structure representing an execution profile with reduced dimensionality. 

46. A method as recited in claim 45, wherein the stopping rule is: extract all 
components whose eigenvalues are greater that a predefined threshold. 

47. A method as recited in claim 45, wherein the stopping rule is: extract those 
components such that the proportion of variation represented by v = — is at least 
equal to a predefined value. 

48. A method as recited in claim 45, fiirfher comprising constructing a matrix 
(P), wherein said matrix is an n x m structure whose rows, p^j , contain values 

showing the degree of relationship of the variation of the f program module and the 
factor or principal component. 

49. A method as recited in claim 48, further comprising the step of forming a 
mapping vector (O) for at least one execution profile vector, wherein the mapping 
vector, O, comprises elements o^. whose values are defined as follows: 

max 

let = Py\ 

\<j<m 

let Oj = index(9^) represent the column number in which the corresponding 
value q. occurs. 

50. A method as recited in claim 49, wherein the mapping vector contains data 
to map probe event frequencies recorded in the execution profile vector onto 
corresponding virtual module equivalents. 
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51 . A method as recited in claim 50, wherein a frequency count for each 
instrumentation point k in an execution profile vector is represented by a value 



and the mapping vector element contains an the index value that^ maps into. 

52. A method as recited in claim 51, wherein m orthogonal sources of 
variation in the data vector D representing the original n program instrumentation 
points are identified. 

53. A method as recited in claim 52, wherein, on each of the original raw 
execution profiles, the instrumentation point firequency count is represented in the 
elements, x- ^ , of the profile vector, X. . 

54. A method as recited in claim 53, wherein the mapping vector contains data 
to map probe event frequencies recorded in the execution profile vector onto 
corresponding virtual module equivalents; and wherein, after the mapping vector has 
been established, a virtual profile vector (Y. ) is employed to contain the fi:equency 
counts for interactions among virtual execution domain sets. 

55. A method as recited in claim 54, wherein the virtual profile vector, X j is 
defined by: 



56. A computer system, comprising: 

(a) a plurality of program modules; 

(b) monitoring means for monitoring transitions between and among 
instrumentation points within the program modules, wherein said monitoring is 
performed by employing software signals obtained from instrumented code in the 



« 



yk,.=T,fi.\.) where. 




0 if Oi^k 
Xj , otherwise * 
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program modules, and for providing program instrumentation trace data representative of 
the transitions between and among program modules within a time frame; 

(c) means for identifying a relatively small set of virtual execution domains 
whose activity is substantially uncorrelated, and using this information to reduce the 
amount of trace data needed to detect anomalous activity; 

(d) means for comparing the reduced amount of trace data with predefined 
data indicative of a nominal operation of the computer system; and 

(e) means for identifying an anomalous operation of the computer system 
based on the result of the comparison. 

57. A system as recited in claim 56, wherein said program execution trace data 
is employed to provide an execution profile including a list of execution paths that 
have executed in a specified time firame and the frequencies of executions. 

58. A system as recited in claim 56, wherein each program module includes a 
mechanism for calling another module, and wherein step (c) comprises the use of a 
statistical method to identify a relativefy small set of cohesive program modules that 
represent dynamic bindings among program modules as they execute. 

59. A system as recited in claim 56, wherein instrumentation points are 
employed to monitor the activity of an executing program and to indicate an epoch in 
the execution of the program. 

60. A system as recited in claim 59, further comprising recording, in a first 
execution profile for tiie program, telemetry from the instrumentation points at each 
epoch. 

61 . A system as recited in claim 60, wherein the first execution profile 
comprises an n element vector (X) comprising at least one entry for each program 
module, and wherein each element, , of said vector contains a frequency count for 
the number of times that the corresponding iastoiimentation point m. has executed 
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n 

during an era of k epochs, where A: = ^ jc. ; and wherein an execution profile is 

recorded whenever the number of epochs, k, reaches a predefined count, at which 
time the contents of the execution profile vector is set to zero. 

62. A system as recited in claim 61, wherein the recorded activity of the 
program during its last L = jK epochs is stored in a sequence of / execution profiles, 
Xj , X2 , . - . , X^. , where the value x^ ^ represents the fl-equency of execution of the 
program module on the execution profile. 

63. A system as recited in claim 62, further comprising the step of reducing 
the dimensionality of the execution profiles from n, the number of instrumentation 
points whose activity is highly correlated, to a smaller set of m virtual instrumentation 
points whose activity is uncorrelated. 

64. A system as recited in claim 63, wherein the statistical technique of 
principal components analysis is employed to reduce the dimensionality of the 
execution profiles. 

65. A system as recited in claim 63, wherein the statistical technique of 
principal factor analysis is employed to reduce the dimensionality of the execution 
profiles. 

66. A system as recited in claim 63, wherein an w x y, 7 > n data matrix 

D = Xi,X2,...,X^ is factored into m virtual orthogonal module components, where m 
is less than n, whereby the dimensionality is reduced fi:om n to m. 

67. A system as recited in claim 66, wherein an eigenvalue /L. is associated 
with each of the m orthogonal components. 
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68. A system as recited in claim 67, wherein the eigenvalues satisfy the 

n 

relation ^/l^ = w . 



69. A system as recited in claim 68, further comprising using a predefined 
stopping rule in determining a number of components extracted in an orthogonal 
structure representing an execution profile with reduced dimensionality. 

70. A system as recited in claim 69, wherein the stopping rule is: extract all 
components whose eigenvalues are greater that a predefined threshold. 

71 . A system as recited in claim 69, wherein the stopping rule is: extract those 

1 

components such that the proportion of variation represented by v = — V /i^. is at least 
equal to a predefined value. 

72. A system as recited in claim 68, further comprising constructing a matrix 
(P), wherein said matrix is an « x w structure whose rows, p^j , contain values 
showing the degree of relationship of the variation of the f program module and the 

factor or principal component. 

73. A system as recited in claim 72, further comprising the step of forming a 
mapping vector (O) for at least one execution profile vector, wherein the mapping 
vector, O, comprises elements whose values are defined as follows: 

max 

\<j<m 

let Oj = index(^^.) represent the column number in which the corresponding 
value qj occurs. 



Page 36 of 38 



SOFT-0004 



PATENT 



74. A system as recited in claim 73, wherein the mapping vector contains data 
to map probe event frequencies recorded in the execution proJBle vector onto 
corresponding virtual module equivalents. 

75. A system as recited in claim 74, wherein a frequency count for each 
instrumentation point k in an execution profile vector is represented by a value , 
and the mapping vector element Oj^ contains an the index value that ^ maps into. 

76. A system as recited in claim 75, wherein w orthogonal sources of variation 
in the data vector D representing the original n program instrumentation points are 
identified. 

77. A system as recited in claim 76, wherein, on each of the original raw 
execution profiles, the instrumentation point frequency count is represented in the 
elements, x^ j , of the profile vector, X, . 

78. A system as recited in claim 77, wherein the mapping vector contains data 
to map probe event frequencies recorded in the execution profile vector onto 
corresponding virtual module equivalents; and wherein, after the mapping vector has 
been established, a virtual profile vector (X ) is employed to contain the frequency 
counts for interactions among virtual execution domain sets. 

79. A system as recited in claim 78, wherein the virtual profile vector, "5^. , is 
defined by: 



n 



1=1 
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